Effects of Question Format on Test-Taker Cognition

Technology-based, interactive test questions are common in large-scale assessments, yet how alternative question formats influence test-taker cognition is not well understood. In a series of studies, we investigated test-taker performance on isomorphic questions using alternative presentation layouts and modes of responding. Adult participants solved math problems in three formats, each of which regularly appear in many large-scale assessments: 1) forced-choice (explicit True-False options) presented in a table format, 2) check-all-that-apply (implicit True-False options) presented in a table format, and 3) check-all-that apply presented as separate questions. Participants’ solution time and affirmative selection rate suggested different cognitive processes for the question formats, particularly when they were uncertain of their answers. We propose a cognitive model to account for the results and predict the impact of alternative question formats on test-takers. We discuss how principles of cognitive science and human-computer interaction provide direct implications for designing assessment questions and understanding test-taker cognition.